The Suffix Sequoia

Standard technologies for sequence searching do not use database indexes. These solutions can be divided into exhaustive algorithms, e.g. the Smith-Waterman algorithm [11], and heuristic ones, like BLAST [1, 2], FASTA [10], and BLAT [7]. Specialised tools for DNA matching exist, such as SIM4 [3] and SSAHA [9]. Only BLAT and SSAHA use indexing. BLAT can be used with proteins, however, its sensitivity is not as good as that of BLAST. To our knowledge, known indexes do not support full sensitivity searching. We showed [6] that the suffix tree produces an indexing gain but does not deliver good performance. Myers and Durbin [8] index the query with regard to the similarity matrix, but not the text itself. Significant effort has been made to use hardware parallelisation [5, 12, 13] and Field Programmable Gate Arrays 1 [4, 14]. Possible hardware solutions complement algorithmic solutions, and are orthogonal to the database approach which uses indexing. We propose a new data structure which combines the features of the suffix tree and of an array-based index, and we measure how well this structure performs in approximate matching. In comparison to [8], where 4% of the DP matrix is computed, and the entire text is scanned, we do not require a full text scan, but exhaustively scan a virtual index, while computing the DP matrix. We access the index on disk only if significant matches are found. In tests with 400 million AAs, and index window size 5, the upper bound on the size of the matrix computation is 1.6% 2 and, in practice, only part of the DP matrix is computed, due to the sparsity of the cost matrix.